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Abstract 

One of the most common problems of scientific applications is com¬ 
putation of the derivative of a function specified by possibly noisy or 
imprecise experimental data. Application of conventional techniques 
for numerically calculating derivatives will amplify the noise making 
the result useless. We address this typical ill-posed problem by appli¬ 
cation of perturbation method to linear first kind equations Ax = / 
with bounded operator A. We assume that we know the operator A 
and source function / only such as ||A — A|| < 5, ||/ — /|| < 5. The 
regularizing equation Ax + B{a)x = / possesses the unique solution. 
Here a G 5, S' is assumed to be an open space in 0 G S', a = a((5). 

As result of proposed theory, we suggest a novel algorithm providing 
accurate results even in the presence of a large amount of noise. 

Keywords; operator and integral equations of the first kind; stable 
differentiation; perturbation method, regularization parameter. 


Introduction 


Let A be bounded operator in banach space X with range R{A) in a 
banach space Y. Let us consider the following linear operator equation 


Ax = f, feR{A). 


( 1 ) 
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We assume domain R{A) can be non closed and KerA 7 ^ {0}. In many 
practical problems one needs to solve an approximate equation 

Ax = /, (2) 

instead of exact equation. Here A and / are approximations of exact operator 
A and right hand side function / correspondingly such as 

||^--4||<5i, ||/-/||<h2 , 5 = max{5i,(52}. (3) 

The problem of solution of the equation (|^ is ill-posed and therefore unsta¬ 
ble even with respect to small errors and it needs regularization in real world 
numerous applications. The basic results in regularization theory and meth¬ 
ods for solution of the inverse problems have been gained in scientihc schools 
of A. N. Tikhonov, V. I. Ivanov and M. M. Laverentiev. Nowadays, this held 
of contemporary mathematics promotes the developments of many interdis¬ 
ciplinary helds in science and technologies [DEI El a EE EH ED- There were 
many efficient regularization methods have been proposed for operator equa¬ 
tion Q. The most efficient regularization methods are Tikhonov’s method of 
stabilizer functional, the quasi-solution method suggested by V. K. Ivanov, 
M. M. Laverentiev perturbation method, V. A. Morozov’s discrepancy princi¬ 
ple and other methods. The principal role in the theory plays variational ap¬ 
proaches, spectral theory, perturbation theory and functional analysis meth¬ 
ods. V.P. Maslov (here readers may refer to [ 8 ]) has established the equiv¬ 
alence of ill-posed problem solution existence and convergence of the regu¬ 
larization process. There is the constant interest of regularization methods 
to be applied in interdisciplinary research and applications related to signal 
and image processing, numerical differentiation and inverse problems. 

In this article we will address the regularized processes construction by 
introduction of the following perturbed equation 

AXa + B{a)Xa = f. (4) 

In this paper we continue and upgrade the results 0. iiBi. la- It is to 
be noted that regularization method based on perturbed equation was hrst 
proposed by M. M. Laverentiev jlj in case of completely continuous self- 
adjoint and positive operator A and B{a) = a. 

Following im we select the stabilizing operator (SO) B{a) to make solu¬ 
tion Xa unique and provide computations stability. Let us call a E S gMA 
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as vector parameter of regularization. Here S is an open set, zero belongs 
to the boundary of this set (briefly, S-sectoral neighborhood of zero in M”), 
lim B(a) = 0. Parameter a we adiust to the data error level 6. Similar 

approach was suggested in the monographs na, mi, but in this article the 
regularization parameter a can be vector. Previously only the simple case 
has been addressed with B{a) = Bq + aBi, a G M"*". Such SO has been 
employed in the development and justihcation of iterative methods of Fred¬ 
holm points Aq calculation, zeros and the elements of the generalized Jordan 
sets of operator functions imi], for the construction of approximate meth¬ 
ods in the theory of branching of solutions of nonlinear operator equations 
with parameters [m El na [15], of construction of solutions of differential- 
operator equations with irreversible operator coefficient in the main part of 
mi- In present article we propose the novel theory for operator systems 
regularization. 

The paper is organized as follows. In Sec. 1 we obtained the sufficient 
conditions when perturbed equation (|^ enables a regularization process. In 
Sec. 2 we suggested the choice of SO B{a). An important role is played 
by a classic theorem of Banach - Steinhaus. In Sec. 3 we consider the 
application of regularizing equation of the form (|^ in the problem of stable 
differentiation. 


1 The fundamental theorem of regularization 
by the perturbation method 

Apart from equations 0 . 0.0 let us introduce the equations 


{Ax + B{a))x = /, 

(5) 

{Ax -f B{a))x = /. 

(6) 

Operator B{a) errors can be always included into the operator A. Equation 
([^ we call regularized equation (RE) for the problem (|^. The following 
estimates are assumed to be fulhlled below 

||(A + R(«))-^||<c(|a|), 

(7) 

\\B{a)\\<d{\a\l 

(8) 
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where c(|a|) is continuous function, a E S G M", 0 G S', lim c(|a|) = 

|a|-5>0 


oo, 
X* ■■ 


lim h(|a|) = 0. Ifa;* is the solution to equation (111), then {A+B{a)) ^f — 

|a|-s>0 

= {A + B{a))~^ B{q)x* . Therefore, we have 


Lemma 1. Let x* he some solution to the equation x{a) satisfy the 
equation Then, in order to Xa -E x* for S 3 a ^ 0 it is necessary and 
sufficient to have the following equality fulfilled 


S{a, X*) = \\{A + B{a)) ^B(a)x*\\^ 0 for S 3 a ^ 0. (9) 

In in] there are sufficient conditions for ensuring estimates 0-(§ , and 
examples addressing case of vector parameter. Application of such estimates 
for solving nonlinear equations are also considered. Let us follow [T2| and 
introduce the following definition. 

Defintion 1. We call the condition 0 as stabilization condition, operator 
B{a), we call stabilization operator ii it satisfy the condition ([^, and solution 
X* we call B-normal solution of equation ([^. 

Remark 1. Obviously the limit of the sequence is unique one in normed 
space and therefore the equation 0 can have only one B-normal solution. 

From estimates 0-(§ it follows 

Lemma 2. Let Xa and Xa be solutions of the equations Q) and 0 corre¬ 
spondingly. If parameter a = a(5) E S is selected such as 5 —)■ 0 


a((5)| -E 0 and 5c(|a((5)|) —)■ 0, 


( 10 ) 


then lim \ \xa — XaW = 0. 

(5-s>0 


Defintion 2. Condition (10) we call the coordination condition of vector 
parameter a with error level 5. 


The coordination conditions play principal role in all regularization meth¬ 
ods for ill-posed problems (here readers may refer e.g. to |2l HH] IH HOl IT^ 
EUniES]). The coordination condition is assumed to be fulfilled. Below we 
also assume a depends on 5 but for the sake of brevity we omit this fact. 
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Lemma 3. Let the estimates 10-0 be satisfied as well as coordination 
condition for the regularization parameter (10). Next we select q G (0,1) and 
find (5 > 0 such as for 6 < 6o the following inequality 

(5c(|a|))<g (11) 

will be fulfilled. Then A + B{a) is continuously invertible operator and the 
following estimates are fulfilled 


||(A + 5(a))-^||< 


1-q 
c(|a|) 


-ii 


( 12 ) 


ll(-4 + B(a))-'/ll < ll(-4 + B{a)Wf\\ + + B(a))-‘/l|. (13) 


Proof. Based on estimate ([^ for V/ we have 


||(A - /l)(A + B{a))-^f\\ < <511(24 + B{a))-^f\\. 


(14) 


Hence taking into account the estimates ([^, ([^, (11), we have the following 
inequality 

\\{A-A){A + B{a))-^f\\<6c{\a\)<q\\f\\. (15) 

Now since q < 1 we have 24 + 5(0!) = {I + (24 — 24 )(24 + 5(o))~^)(24 + Bja)), 
then existence of inverse operator (24 + B{a))~^, as well as estimate (12) 
follows from known inverse operator Th.. Next we employ the following 
operator identity C~^ = D~^ — D~^{I + {C — D)D~^)~^{C — D)D~^ where 
C = (24 + 5 ( 0 )), D = 24 + 5 ( 0 ) and, based on inequalities (14), (15) we get 
estimate (13). 

□ 

Theorem 1 (Main Theorem). Let conditions of Lemma [g] be fulfilled, i.e 
parameter a is coordinated with noise level 5. Then RE Jm has a unique 
solution Xa- Moreover if in addition x* is solution of the exact equation 0 . 
then the following estimate is fulfilled 




< S'(o, X*) + 


Ml«l) 

1 - q 


1 + ||a;*|| + S{a,x*) 


(16) 


If also X* is B-normal solution of the equation 0 then {xa} converges to x* 
at a rate determined by bound (16) as 6 ^ 0. 
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Proof. Existence and nniqueness of the seqnence as solution of RE 

for a G S' proved in Lemma ^ Since {A + B{a)){xa — x*) = f — 


f — {A — A)x* — B{a)x*, then we get the desired bound (16) ||xq, — x*|| < 

\\{A+B{a))-mf-f\\ + \\{A-A)x*\\ + \\{A+B{^^^^^ 


1 + ||x*|| + S{a,x*) ) based on the proved estimates ( 12 ), (13) and 


< ^ 


a, x*) + 


l-q 

(8). Since x* is R-normal solution then lim S(q!,x*) = 0. And thanks to 

Q. —^0 

parameter a coordinated with noise level 6 we have lim(5c(|a|) = 0. Hence, 

S —>-0 

due to bound (16) lim ||xa — x*|| =0 which completes the proof. 

_J 5—>-0 

□ 


As footnote of the section it’s to be mentioned that for practical applications 
of this theorem one needs recommendations on the choice of SO B{a) and B- 
normal solution existence conditions. It’s also useful to know the necessary 
and sufficient conditions of the existence of R-normal solutions x* to the 
exact equation ([^. These issues we discuss below. 


2 


Stabilizing operator B{a) selection, B-normal 
solutions existence and correctness class of 


problem (1) 


If A is Fredholm operator, {0j}i is basis in Mi^A), {V’iji is basis in J\f*{A), 
then (here readers may refer to Sec. 22 in textbook [ 22 ]), one may assume 

n 

where { 7 *}, {zi} are selected such as det[{(j)i,'jk)]lk=i A 0 , 

i=\ 

/ {l iii = k 

= < herewith the equation 

^ ' \{) iii^k 


n 

Ax = f - '^{f,'ipi)zi (17) 

i=l 

is resolvable for arbitrary source function /. Let us now recall /, which is 

n _ 

^-approximation of /. Then perturbed equation Ax + '^{x,''yi)zi = f — 

i=l 

n _ 

unique solution x such as ||x — x*|| —)■ 0 for 5 —)■ 0 , where 

i=l 


6 



















X* is unique solution of exact solution (17) for which (a:*,7*) = 0, i = l,n. 
Thus, in the case of a Fredholm operator A as a stabilizing operator one 


can take hnite-dimensional operator B = which does not depend 

1 

on the parameter a. That is the regularization of iterative methods we em¬ 
ployed in our papers inii for the second order nonlinear equations 

studies with parameters. Of course, with this choice of SO B it is required 
to have information about the kernel of the operator A and its defect sub¬ 
space. Therefore, it is of interest to give recommendations on the choice of 
SO B{a) without the use of such information. It is important to consider in 
a more complex problem solving the hrst kind equations, when the range of 
the operator A is not closed. It is to be noted that in papers [iHl HD] and in 
the monograph [lainj we constructed the SO for the hrst kind equations as 
Bo + aBi where a G M"*". Below we consider the generalization of such results 
when B = B{a), a G S' C M”. Our previous results presented in papers 
promsiE] follows from the proved Th. 2 and 3 as special cases. 


Theorem 2. Let ||(74-|-i?(Q;)) ^|| < c(|a|), ||i?(a)|| < d{\a\) fora G S' C M”, 
c(|q!|), ddal) are continuous functions, lim c(|a|) = 00 , lim d(|a|) = 0. Let 

| o |->0 | a |^0 

lim c(|a|)(i(|Q!|) < 00 , Af{A) = 0, R{A) = Y. Then unique solution x* of 

\ a \^0 

equation 0 is B-normal solution and operator B{a) is its SO. 


Proof. First, lets B{a)x* G M(A) for a G S'. Then exists element Xi{a) such 
as Axi{a) = B{a)x* . Then {A + B{a))~^B{a)x* = {A + B{a))~^{Axi{a) + 
B{a)x\{a) — B{a)xi{a)) = xi{a) — {A + B{a))~^B{a)xi{a). Since 5(0) = 
0, N^A) = {0} then lim xi{a) = 0. It is to be noted that by con- 

S^ol —^0 

dition IKA -I- B{a))~^B{a)\\ < c{\a\)d{\a\), where c(|Q!|)(i(|Q!|) is continu¬ 
ous function such as the limit lim c(|Q!|)(i(|a|) is hnite. Then a-sequence 

|a|-s>0 

{||(74-|-5(Q!))“^5(a)x*||} inhnitesimal when S' 9 a —)■ 0. The sequence of op¬ 
erators {{A + B{a))~^B{a)} converges pointwise to the zero operator on the 
linear manifold Lq = {x\ B{a)x G ^(A)}. Thus, we have proved that the Th. 
is true when B{a)x* G 5(^4). Because by condition sup c(|a|)(i(|a|) < 00 then 

a-sequence {IKA -|-5(a))“^5(Q!)||} is bounded. Therefore, the sequence of 
linear operators {(A + B{a))~^B{a)} in space X converges pointwise to the 
zero operator on the linear manifold Lq = {x]^B{a)x G R{A)}. But then, 
on the basis of the Banach - Steinhaus Theorem we have pointwise conver- 
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gence of this operators sequence to the zero operator and on the closure Lq, 
i.e. when B{a)x* G R{A). Since R{A) = Y and B{a) G C{X —)■ Y), then 
B{a)x* G Y and Th. 2 is proved. 

□ 

The conditions of Th. 2 can be relaxed (here readers may refer to Cor. 1 
and Th. 3). 

Corollary 1. If R{A) C Y, lim xi{a) = 0 then solution x* of exact equa- 

S9Q-5-0 

tion is B-normal iff B(a)x* G R{A). 

Remark 2. In Cor. 1 condition M{A) = {0} is not used. The set L = 
{x\B{a)x G R(^)} in conditions of Cor.1 defines maximum correctness class. 

It is to be noted that in Th. 2 we used the assumption on the hnite limit 
lim IKA + R(a))“^||||i?(a)||. We can also relax this limitation. 

Theorem 3. Let ||(A + ai?)“^|| < c(a), where a G c{a) : (0,ao] 
is continuous function. Suppose that there is a positive integer n > 1 such as 
lim c(a)a® = cx), i = 0, n — 1, lim c(a)a"' < oo. Let Xq satisfies the equation 

a—>-0 a—^-O 

0 and in case of n > 2 there exist xi, ■ • • ,Xn-i which satisfy the sequence 
of equations Axi = Bxi-i, i = 1, ■ ■ ■ ,n — 1. Then xq is B-normal solution 
to equation l^iffBxn-ieR{A). 

Proof. Since Axi = Bxi-i we have an equality {A + aB)~^aBxo = a{A + 
aB)~^{Axi + aBxi — aBxi) = axi — a^i^A + aB)~^Bxi = ■ ■ ■ = ax\ — 

0 ^X 2 H- {—lYa^{A + aB)~^Bxn-i, where if a —)■ 0 then hrst n — 2 terms 

located on the right hand side are inhnitesimal. Using the Banach-Steinhaus 
Theorem lets make sure that {a^'IKA + aB)~^Bxn-i\W is inhnitesimal. In¬ 
deed, if Bxn-i G M(y4), then there exist x„ such as Axn = Bxn-i- But 
in this case a^{A + aB)~^Bxn-i = a'^{A -|- aB)~^{A -|- aB — aB)xn = 
a”x„ — a'^^^{A +aBYBxn-, where a^^^\\{A + aBYBxn\ \ < Q!"''''^c(q!)||Rx„||, 
lim a'^+^c(a) = 0. Consequently, {Wa'^^A + aBYBxn-i\\} is inhnitesimal, 

a—>-0 

and the sequence of linear operators {a"‘{A + aBYB} pointwise converges to 
zero operator on the linear manifold L = {x|Rx G R(^)} and the sequence 
{Wa^^AY aBYB\\] is bounded. Since J = {x | Bx G R(^)} we complete the 
proof by the reference to the Banach-Steinhaus Theorem. 

□ 

We apply Th. 2 for construction of the stable diherentiation algorithm below. 











3 On differentiation regularization 


Let 1/ : J C M —?• M be continuous and differentiable function on the interval 
I, and let its derivative y'{t) be continuous on the interval (a, 6) C I. Then 
y(t) — y{+a) — y'{+a)(t — a) = o(t — a) as t —)• +a. Let y : [a, 6] —)• M be a 
bounded function and values c, d such as sup |/(t) — f{t)\ = 0{6), where 

a<t<b 

fit) = y{t) — y{+a) — y'{+a){t — a), f = y{t) — c — d{t — a). In applications 
the values yitf) are usually know for ti = ih & [a, b] such as \yiti) — yiti)\ = 
0{6). Our objective here is to hnd y'(L) with accuracy up to e. Stable 
differentiation problem attracted many scientists including A. N. Tikhonov, 
V. Ya. Arsenin, V. V. Vasin, V. B. Demidovich, T. F. Dolgopolova and 
V. K. Ivanov. Here readers may refer to textbook [2l], p. 158 and [1] for 
review of recent results . Let us introduce the following equations: 

x{s)ds = f{t), (18) 



Xa{s) ds + aXa{t) = f{t), / Xa{s) ds + aXa{t) = f (t). 


Therefore here we have A := J^[-] ds, R{A) = 


^ /(+«) - 0 [> -: C [a 


( 1 ) 


R{A) = C[a,fe], B{a) := al, X = Y = C[a,b]- We construct the inverse 
operator [A + al)~^ G C{C\^a,h] ^[a,6]) explicitly as [A + al)~^ ~ a ~ 


^ la ^ “ ['Ids. Since f(t) E R(A) then equation (18) has unique solution 

x*(t) = A~^f = y'{t) — y'{+a). It is to be noted that 


\\{A+aI) < - (^1 + - 


max / 

a<t<b j^ 

— — w <2 


. 1 , _ a — h 2 

e “ as = —(2—e “ ) < —. 

a a 


Parameter a should be coordinated with 5, e.g. if a = \/~5. Since ||H(a)|| = a, 
c{a) = N^A) = {0} then based on Th. 2 continuous function x*{t) = 


( 1 ) 


y'{t) —y'{Ya) is H-normal solution iff x*{t) E R{A). In our case R{A) = C [a,b], 


( 1 ) 


. Taking into account that linear functions space C [a,6], is dense Li = {f{t) E 
^[a,b], /(+o) = 0}, x*{+a) = 0, then x*{t) E R{A). Therefore based on the 
Main Theorem and Th. 2, the formula 


Xa{t) = 


y{t) 


d{t 


a 


-^ / e {y{s) - c - d{s - a)) ds (19) 




9 







defines algorithm of stable differentiation y'{t). Or, more precisely Ve > 0 

3(5o = <5 o(£) > 0 such as, if sup |/(t)—/(f)| < 5, 5 < 5o(^) then max \xa{t) — f'{t)\ < 

a<t<b a<t<b 

e. If we select a = y/d, then lim max \xa{t) — — 2 /'(+a))| = 0. There- 

5^0 a<t<b _ 

fore, {xa} converges uniformly to y'{t) — y'{+a) as 5 —>■ 0. Based on (19) 
we constructed regularized differentiation algorithm, which is uniform w.r.t. 
t G [a, b]. Let us demonstrate its efficiency below. 


4 Numeric examples 

In this section we included two examples to demonstrate the efficiency of our 
approach. We add noise to exact data as y{t) = y{t) + 5R{t) with the noise 
levels 5=0.1, 5=0.01 and 5=0.001, where R{t) is a random function with zero 
mean value and standard deviation a = 1. The number of grid points used 
is 512. Trapezoidal quadrature rule is used. 

Example 4.1. For the first example we use the function y{t) = ^ 3 ^ sin (^) , t G 
[0,3], with its derivative y'(t) = ^^=^sin(^) + cos (^) . Fig. 1 

demonstrates exact and computed derivatives and the errors, noise level 5=0.001. 
The maximum errors are given in Tab. 1. 

Example 4.2. Here we used the exact function y{t) = cos(y)e“*^, t G [0, 5], 
with its exact derivative y'(t) = —sin(y) + 2tcos(^)). The maximum 
errors for this example are shown in Tab. 2. 
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(a) 



(b) 


Figure 1: Example 1. (a) the exact and the computed derivatives y'{t). (b) 
the errors. 

The noise level 5=0.001 is used to generate y{t). 


Table 1 


5 

0.1 

0.01 

0.001 

max error 

0.172977091 

0.284849645 

0.70584444 
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(a) 



(b) 


Figure 2: Example 2. (a) the exact and the computed derivatives y'(t). (b) 
the errors. 

The noise level 5=0.001 is used to generate y{t). 


Table 2 


5 

0.1 

0.01 

0.001 

max error 

0.169489112 

0.277548421 

0.637657088 
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